Our experience with the visual world guides how we understand it. The regularity of our experience of objects within environments and with each other provides a rich context that influences our perception and recognition of objects and categories (Bar, 2004; Greene, 2013). We do not often see faces in isolation, rather our experience of faces is embedded within a contextually rich environment – a face is usually conjoined to a body – and this regular structure influences how we perceive and interact with faces (Bar and Ullman, 1996; Meeren et al., 2005; Rice et al., 2013). However, it is generally found that neurons critical for the perception of faces (Afraz et al., 2015; Afraz et al., 2006; Moeller et al., 2017) do not respond to bodies, and vice versa (Freiwald and Tsao, 2010; Freiwald et al., 2009; Kiani et al., 2007; Tsao et al., 2006), suggesting that such neurons code for the intrinsic features of either faces or bodies but not both (Chang and Tsao, 2017; Issa and DiCarlo, 2012; Leopold et al., 2006). These studies typically probe neural function by presenting objects in isolation, independent of their typical context.

Here, we reconcile this apparent discrepancy between behavior and neural coding. We recorded from two face-selective domains within inferotemporal cortex (IT) in two macaque monkeys while they viewed complex natural images. By considering the spatial relationships of objects in a natural scene relative to a neuron’s receptive field, we show that face-selective IT neurons respond not just to an image of a face, but to parts of an image where contextual cues—such as a body—indicate a face ought to be, even if what is there is not a face. Thus, we find that face cells demonstrate a large contextual surround tuned to entities that normally co-occur with faces. These results reveal surprisingly complex and heterogeneous inputs to face-selective neurons, suggesting that IT neurons do not represent objects in isolation, but are also sensitive to object relationships that reflect statistical regularities of the experienced environment.

We recorded simultaneously from face-selective neurons in the fMRI-defined middle lateral (ML) and posterior lateral (PL) face patches in one rhesus macaque monkey, Monkey 1, and from ML in another monkey, Monkey 2, using chronically implanted 32-channel multi-electrode arrays (Figure 1—figure supplement 1). When mapped using ~2°x2° faces and round non-face objects, the activating regions were small (1-3°), located in the central visual field (within 2° of fixation, and largely contralateral; Figure 1—figure supplement 2). When tested with images of faces, hands, bodies (only for ML arrays), and objects presented centered on this activating region, all the visually responsive sites in both monkeys responded more to faces than to any other category (Figure 1—figure supplement 3). Category selectivity was assessed with an index (category1-response –category2-response)/ (category1-response + category2-response). In particular, in both monkeys’ ML face patches, responses to faces were much larger than to non-face images (mean channel face vs. non-face selectivity indices = 0.63 +/- 0.06 STD and 0.64 +/- 0. 14 for Monkey’s 1 and 2) or to bodies (mean channel face vs. body selectivity indices = 0.77 +/- 0.11 and 0.60 +/- 0.21). Responses to bodies were weak (body vs. object selectivity indices = −0.54 +/- 0.17 STD and 0.04 +/- 0.27 for ML in both Monkeys 1 and 2). Such a strong preference for faces over other categories is consistent with many previous studies (Bell et al., 2011; Tsao et al., 2006). The array in Monkey 1’s PL face patch was less face selective (mean channel face vs. non-face selectivity indices = 0.24 +/- 0.07 STD) and responses to hands were weak (mean channel hands vs. objects indices = 0.04 +/- 0.02 STD), also consistent with previous work (Issa and DiCarlo, 2012).

To explore how response properties of face neurons represent naturalistic scenes, we mapped the spatial organization of face-selective neurons by presenting complex, familiar and unfamiliar scenes at various positions relative to each site’s activating region (Figure 1). Conventionally IT neurons are studied by presenting single-object images at a single position while the animal holds fixation (Figure 1—figure supplement 3). Here instead, while the animal held fixation, we presented large 16° x 16° complex natural images at a series of different positions, such that different parts of the image were centered over the receptive field across presentations (Figure 1), in order to resolve the discrete spatial pattern of responses for each site to different parts of the image. The recording site illustrated in Figure 1C responded most strongly to the parts of that image that contained monkey faces. We had 28 to 32 visually responsive sites in each of the three implanted arrays. Within each array the responses were all face selective (Figure 1—figure supplement 3) and showed similar receptive-field locations and spatial response patterns to complex scenes (Figure 1—figure supplements 2 and 4). Given the uniformity of responses across channels in each array, we combined all visually responsive sites in each array for all subsequent analyses to improve signal to noise and present population-level results. Importantly, our findings are not contingent on individual unit/recording site effects. Figure 1 shows the spatial pattern of responses to a series of images from the PL array in Monkey 1, and from the ML arrays in Monkeys 1 and 2. Consistent with the PSTHs of image categories presented in isolation (Figure 1—figure supplement 3), all three arrays showed responsiveness that was spatially specific to the faces in each image. Consistent with prior physiological and anatomical work suggesting that face patches are organized into a hierarchy that varies along the posterior-anterior axis (Grimaldi et al., 2016; Moeller et al., 2008), ML recording sites were more face selective and had longer latencies compared to the PL sites during simultaneous recordings (Figure 1—figure supplement 5). Activity specific to the eyes is apparent in both PL and ML (Figure 2). Activity encompassing the rest of the face emerges at longer latencies. This is consistent with prior work showing a preference for the contralateral eye in area PL that emerges between 60–80 ms post stimulus onset and longer latencies for images without a contralateral eye (Issa and DiCarlo, 2012). A similar shift in response latencies was evident when replacing the eyes with a visible occluder, the background of the image, and surrounding texture of the face (Figure 2—figure supplement 1). In our recordings, both PL and ML exhibited a shift in the latency for activity corresponding to faces without eyes, but PL activity still preceded ML (Figure 2; Figure 2—figure supplement 1), indicating that in the absence of eyes, face-specific activity still emerges across the ventral visual hierarchy in a bottom-up fashion.

Figure 1 with 5 supplements see all

Download asset Open asset

Figure 2 with 1 supplement see all

Download asset Open asset

To explore contextual associations of natural face experience, we then presented complex scenes that lacked faces, but contained cues that indicated where a face ought to be. When presented with images in which the face was occluded by some object in the image, surprisingly, the face cells in ML in both monkeys responded to the part of the image where the face ought to be, more than to non-face parts of the images (Figure 3). Though noisier than the population average, responses to occluded faces were apparent in individual channels (Figure 3—figure supplement 1). The magnitude and timing of responses to occluded faces varied across stimuli. Across all stimuli, the population-level activity to occluded faces was reliably higher than the average response to the rest of each image (t-test across 12 and 10 images for Monkeys 1 and 2, respectively; p<0.05, FDR-corrected). Interestingly, the responses to occluded faces on average were slower than the responses to intact faces by ~30–40 ms (Figure 3—figure supplements 2 and 3). The latency of responses to occluded faces was similar to that to eyeless faces (Figure 2; Figure 2—figure supplement 1). By 119 ms and 139 ms post stimulus onset for Monkey 1 and Monkey 2, respectively, the responses to occluded faces were significantly larger than the responses to the rest of the image. In contrast, responses to non-occluded versions of these faces were significantly larger than the responses to the rest of the image starting at 92 ms and 97 ms post stimulus onset for Monkey 1 and Monkey 2, respectively. It is possible that the face cells in ML responded to some tiny part of the head or neck that was still visible in each of the occluded-face photos, though similar visual features were present in other parts of each image that did not evoke strong firing. It is also unlikely that the occluding objects contained visual features that alone evoke strong firing as those objects were present in other parts of each image that did not evoke strong firing (e.g., the section of the rug in-between the two faces and boxes covering the face of the man walking down the stairs vs. boxes below the face). Furthermore, when presented with people wearing masks, baskets or paper bags covering their faces, cells in ML responded to the masked faces (Figure 4). Similar to the occluded faces, responses to masked faces were also apparent in individual channels (Figure 4—figure supplement 1). The population-level responses to masked faces varied in magnitude and timing but were consistently stronger than average responses to the rest of each image (t-test across 10 and 11 images for Monkeys 1 and 2, respectively; p<0.05, FDR-corrected). The latency of responses to masked faces (Figure 4—figure supplements 2 and 3) was also similar to that to eyeless faces (Figure 2; Figure 2—figure supplement 1). Face cells in PL also responded more to occluded faces than to the non-face parts of the images (Figure 4—figure supplement 4; p<0.05, FDR corrected). Though cells in PL also responded on average more strongly to masked faces than to other parts of each image, this difference was not significant (Figure 4—figure supplement 4; p>0.05, FDR corrected). Thus, responses to occluded faces were more prominent at later stages of the ventral visual hierarchy.

Figure 3 with 3 supplements see all

Download asset Open asset

Figure 4 with 4 supplements see all

Download asset Open asset

To ask whether the face-cell responses to masked or occluded faces were due specifically to the context of a body just below the occluded face, we manipulated some images so that a non-face object was present in an image twice, once above a body, and once not (Figure 5, top). In both monkeys’ ML patches, responses to the non-face object positioned above a body were larger than responses to the same object when not above a body, both at the population level (Figure 5) and in individual channels (Figure 5—figure supplement 1). Similar to the occluded and masked faces, population-level responses to these face-swapped images varied in magnitude and latency, but responses to non-face objects were consistently stronger when positioned above a body than when positioned elsewhere (t-test across 8 and 15 images for M1 and M2, respectively; p<0.05, FDR-corrected). Similar to the occluded and masked faces, the latency of responses to the non-face object above a body (Figure 5—figure supplements 2 and 3) was comparable to that to eyeless faces (Figure 2; Figure 2—figure supplement 1). Though cells in PL also responded on average more strongly to objects on top of bodies, this difference was not significant (Figure 4—figure supplement 4; p>0.05, FDR-corrected). Thus, responses in ML to a non-face image were facilitated by a body below it.

Figure 5 with 3 supplements see all

Download asset Open asset

Given our finding that bodies facilitated face-cell responses to masked, occluded, and non-face objects positioned above them, we might expect bodies to also facilitate responses to faces appropriately positioned above bodies, but this was not the case. Instead, responses to faces above bodies were indistinguishable from responses to disembodied faces (Figure 6, leftmost; Figure 6—figure supplements 1 and 2, top two rows). This is consistent with the response maps to a subset of images that showed strong responses to faces regardless of the spatial relation to bodies (columns 3–5 in Figure 5, top). In contrast, responses to face-shaped noise patches (i.e. where face pixels were replaced by random 3-color channel pixel values), face outlines, and non-face objects were facilitated by the presence of a body positioned below (Figure 6; Figure 6—figure supplements 1 and 2). Face-shaped noise patches and face-shaped outlines both elicited responses above baseline even in the absence of the body (middle two columns). This was likely due to these images containing typical shape features of a face, which is sufficient to drive these neurons (Issa and DiCarlo, 2012; Tsao et al., 2006). Importantly, the responses to these images when placed above a body were significantly larger (p<0.05, FDR-corrected). Further, non-face objects activated these neurons only when placed above bodies (Figure 6 rightmost column, p<0.05, FDR-corrected; Figure 6—figure supplements 1 and 2, bottom row). Together, these data suggest a ceiling effect--that bodies facilitate responses of face cells only when the presence of a face is ambiguous. Further, these neurons respond to the region above a body even if no foreground object is present (Figure 6—figure supplement 3). In the absence of a face, the presence of a body below the receptive field appears to be sufficient for driving these face-selective cells.

Figure 6 with 3 supplements see all

Download asset Open asset

If bodies positioned directly below the activating region facilitate face-cell responses, is it because face cells have a discrete excitatory region, below the activating region, that is simply responsive to bodies? Or is the effect more complex, reflecting contextual information that a body should be accompanied by a face? To address this question, we mapped responsiveness to bodies, with and without heads, at varying orientations and locations in a 9 × 9 grid pattern (Figure 7—figure supplement 1). For bodies-with-heads, responses were maximal when the face was centered in the cells’ activating region, for each orientation (Figure 7A, top row). Headless bodies produced activation maps that were similarly maximal at positions in which a face ought to be centered on the cells’ activating region, not just when the body was positioned below the activating region (Figure 7A, bottom two rows). The magnitude of firing rate varied across the tested visual field locations (e.g., firing rates tended to be higher in the upper half of the image). Composite maps show the preferred body orientation for each spatial position tested (Figure 7A, right side). In experiments both with and without heads, preferred orientation systematically varied such that the preferred body orientation at that spatial location positioned the face, or where the face ought to be, within the RF center. Interestingly, these neurons were also responsive when the feet landed in the activating region, for the bodies without heads, and, to a lesser extent, the bodies with heads. This was especially apparent for inverted bodies, suggesting that both end-of-a-body and above-a-body are cues that a face might be expected. The surprising result that face cells are maximally responsive to where a face would be expected, largely irrespective of the body orientation, indicates that the responses to occluded faces documented in Figures 3 and 4 are not responses merely to bodies located below the cells’ face-selective regions, but instead reflect information about where a face ought to be, for a given body configuration.

Figure 7 with 3 supplements see all

Download asset Open asset

If bodies convey information to face cells about where a face ought to be, are there preferred face-body configurations? When face and face-shaped noise were presented attached to bodies in upright and inverted configurations, the strongest response was when the face (or face-shaped noise) was presented at the center of the activating region (Figure 7B). For intact faces with bodies presented at the center of the activating region, there was only a small inversion effect (Figure 7B; whole bodies). While reaction time differences between upright and inverted faces are robust and reliable across studies (Leehey et al., 1978; Yin, 1969), differences in response magnitudes within IT face domains are modest or not consistently found in humans (Aguirre et al., 1999; Haxby et al., 1999; Kanwisher et al., 1998) or monkeys (Bruce, 1982; Pinsk et al., 2009; Rosenfeld and Van Hoesen, 1979; Taubert et al., 2015; Tsao et al., 2006). The lack of a strong inversion effect for intact-faces-over-bodies compared to a stronger inversion effect for non-face-objects-above-bodies could reflect a ceiling effect, or it could mean that, in the presence of a face, tuning is more flexible and more invariant to spatial transformations. In contrast to responses to intact faces, there was a large, sustained difference in response magnitude between upright and inverted face-shaped noise patches with bodies (Figure 7B; noise atop bodies), suggesting that the body facilitation is maximal when the presence of a face is ambiguous and above a body. The preference for upright vs. inverted configurations with the face-shaped noise is consistent with biases apparent in Figure 7A. Though face cells in ML were responsive to headless bodies at a variety of spatial positions, activity tended to be strongest when bodies were positioned below the activating region (Figure 7A; bottom two rows). This was particularly clear in Monkey 2 where the part of the image that gave the strongest response for the inverted (180-degree) body orientation was the feet, meaning that the inverted body still drove neurons most when it was positioned just below the activating region. Thus, for face-ambiguous stimulation, contextual responses are strongest to spatial configurations that reflect typical experience.

How can face-selective neurons respond to bodies positioned below their receptive fields? One possibility is that such contextual sensitivity is intrinsic to IT and pre-specified for the processing of biologically important information. Alternatively, given that the development of face domains depends on seeing faces during development (Arcaro et al., 2017), these conjoint representations might also be learned over development as a consequence of frequently seeing faces and bodies together. To explore whether learning is sufficient to link face and body representations, we analyzed the representation of image categories in a computational model that has similar architectural features to the object processing pathway in primates. Specifically, we took a convolutional neural network, AlexNet (Krizevsky et al., 2012), trained to classify natural images into 1000 object categories, and assessed its activation patterns evoked by images of isolated objects from 6 different categories; human faces, headless human bodies, animals, houses, inanimate objects, and handheld tools (Figure 8A, top). We tested a stimulus set widely used for localizing category-selective regions in IT cortex (Downing et al., 2006). We assessed the similarity of activation patterns between all image pairs in every layer of AlexNet and in pixel space. As seen in the multidimensional scaling visualization, images are distinguished along categorical distinctions even based on just pixel intensity (Figure 8A, bottom). The Euclidean distances between image pairs within each category were smaller than between categories, indicating that images were more homogenous within than between categories and highlighting that low-level features such as shape, color, or texture are a major component of these commonly tested ‘semantic’ categories (also see Dailey and Cottrell, 1999). Distinctions between categories were present in early layers of AlexNet (Figure 8A, bottom). The distinction between faces and non-faces persisted across all layers of AlexNet. The mean distance between face pairs (red) was smaller than the mean distance between faces and non-body, non-face categories (animals, houses, objects, and tools). Across all rectified linear unit (relu) layers of AlexNet, activation patterns were more similar for within category comparisons of both faces and bodies (Figure 8B, solid red and green circles, Spearman rank correlations) than between faces and other categories (dual colored circles). At the pixel level and in early layers of AlexNet, the mean similarity between faces and bodies (half red/half green circles) was greater than the similarity between faces and the other categories; this similarity was also stronger to that obtained under randomization of the stimulus labels for non-face categories (this randomization distribution is marked by the grey shaded region that denotes 2.5% and 97.5% bounds). In the deeper layers of AlexNet (relu6 and relu7), face and body image representations become more overlapping (Figure 8B, inset). In relu7, the correlation between face and body image pairs were only slightly weaker than the correlation between pairs of body images. Comparable effects were observed when assessing similarity with Pearson correlations and Euclidean distance on z-score normalized data. Further, these results were specific to the trained AlexNet, as untrained versions of the network did not yield strong face-body correlations and these correlations were no different than correlations between faces and other categories (Figure 8—figure supplement 1). These results demonstrate that representations of faces and bodies can merge over learning without any explicit training about faces and bodies. This is particularly impressive given that AlexNet was not trained to categorize faces, bodies, or even humans. Because faces and bodies co-occur in people and people were present in the training images, faces and bodies share variance in their informativeness with respect to class labels. There is thus pressure for the network to use overlapping representations in the face and body hidden layer representations in this supervised learning scenario. This phenomenon should hold more generally in unsupervised or self-supervised settings as the presence of image co-occurrence in the input would be expected to lead to similar overlap. Though convolutional networks can capture some aspects of the representational structure of biological networks (Hong et al., 2016; Ponce et al., 2019), these networks as currently implemented are not well suited for probing the retinotopic specificity of object representations, because they are explicitly constrained to process inputs from different portions of an image in the same way. Future work could probe the spatial relationships between commonly experienced object categories in computational models more optimized for maintaining spatial coding.

Figure 8 with 1 supplement see all

Download asset Open asset

Our finding that face-selective neurons respond to parts of an image where a face ought to be—above a body—goes well beyond previous observations on category selectivity in inferotemporal cortex. Previous neuronal recordings in macaques found that face-selective neurons respond only weakly to images of headless bodies, and body-selective neurons poorly to faces, suggesting that face and body circuits are largely independent (Freiwald and Tsao, 2010; Freiwald et al., 2009; Kiani et al., 2007; Tsao et al., 2006). Human fMRI studies have also provided evidence that face and body processing are independent (Kaiser et al., 2014; Peelen and Downing, 2005; Peelen et al., 2006; Schwarzlose et al., 2005) and that face-selective domains in the FFA do not respond to headless animals or human bodies presented at fixation (Kanwisher et al., 1999; Peelen and Downing, 2005; Schwarzlose et al., 2005). Consistent with these prior findings, the face neurons we recorded from responded strongly and selectively to faces presented over a limited part of the visual field but did not respond to bodies presented to the same region. However, these neurons also responded to masked and occluded faces that lacked intrinsic facial features, but were above bodies. Further, when non-face objects that did not by themselves evoke a response from these neurons were placed above a body, the face-selective neurons fired robustly. Consistent across all six experiments, these neurons responded to bodies positioned such that the body implied the presence of a face within the neurons’ activating regions. In prior studies, both headless bodies and faces were presented at fixation, so our results suggest that these studies might have found larger responses to bodies if the bodies had been presented lower in the visual field, below the center of maximal activation to faces. Thus, face cells demonstrate a large contextual surround tuned to entities that normally co-occur with faces.

Though prior studies have mostly provided evidence for the independence of face- and body- processing circuits, a few fMRI studies found facilitatory influences of bodies on blood-flow responses in face-selective brain regions: The face fusiform area (FFA) in humans has been reported to be responsive to animal bodies with obscured faces (Chao et al., 1998) or to blurred faces above bodies (Cox et al., 2004). Cox et al., 2004 also found body-below facilitation of BOLD signal for blurred faces but not for clear faces, and they found that the BOLD signal facilitation depended on the face and body being in a correct spatial configuration. Both these findings are consistent with our neuronal recording results. In monkeys, the most anterior face patches show stronger fMRI activations to images of whole monkeys than to the sum of the responses to faces and to bodies; however, the middle and posterior face patches (which we recorded here) do not show facilitation, and have only sub-additive responses (Fisher and Freiwald, 2015). Thus, although there is some fMRI evidence for interactions between face and body processing, fMRI studies are limited in that they can measure responses only of large populations of neurons of potentially mixed selectivity and therefore cannot resolve the relationship of face and body neural circuits. Here we show by direct neuronal recording that face and body circuits are not independent and that face-selective neurons can express large, complex, silent, contextual surround effects. From simultaneously recording neurons in PL and ML face patches, we show that these responses emerge fast and are first detected in the posterior patch, suggesting that contextual effects involve at least some feedforward inputs that cannot be attributed to cognitive processes such as mental imagery or expectation. The idea that the context of a visual scene leads to the expectation, enhanced probability, or prior of seeing some particular item is usually thought to involve top-down processes. However, if the feedforward inputs to, say, face-selective neurons, are sculpted by experience to be responsive not only to faces, but also to things frequently experienced in conjunction with faces, then such expectations would be manifest as the kind of complex response properties reported here. Furthermore, in contrast to the traditional view of a dichotomy of spatial and object vision, our data demonstrate that object processing in IT is spatially specific (also see Op De Beeck and Vogels, 2000).

More broadly, these results highlight that neurons in inferotemporal cortex do not code objects and complex shapes in isolation. We propose that these neurons are sensitive to the statistical regularities of their cumulative experience and that the facilitation of face-selective neurons in the absence of a clear face reflects a lifetime of experience that bodies are usually accompanied by faces. Recently, we proposed that an intrinsic retinotopic organization (Arcaro and Livingstone, 2017; also see Hasson et al., 2002) guides the subsequent development of category-selectivity in inferotemporal cortex, such as face and body domains (Arcaro et al., 2017). For example, monkeys normally fixate faces, and they develop face domains within regions of inferotemporal cortex that represent central visual space (Figure 9, top). Here, we propose that these same developmental mechanisms can account for contextual learning. Our results indicate that, just as the retinotopic regularity of experiencing faces guides where in IT face domains develop, the spatial regularity between faces and bodies is also learned in a retinotopic fashion and may account for why face and body patches stereotypically emerge in adjacent parts of cortex (Figure 9, bottom) in both humans (Schwarzlose et al., 2005) and monkeys (Bell et al., 2009; Pinsk et al., 2009). Such a spatial regularity is consistent with prior observations of a lower visual field bias for the RF centers of body-selective neurons near ML (Popivanov et al., 2015) and an upper visual field bias for eye-selective neurons in PL (Issa and DiCarlo, 2012). Thus, something as complex and abstract as context may be firmly grounded in the ubiquitous mechanisms that govern the wiring-up of the brain across development (Arcaro et al., 2019).

Figure 9

Download asset Open asset

All procedures were approved in protocol (1146) by the Harvard Medical School Institutional Animal Care and Use Committee (IACUC), following the Guide for the care and use of laboratory animals (Ed 8). This paper conforms to the ARRIVE Guidelines checklist.

Custom scripts and data to reproduce figures have been deposited to GitHub (https://github.com/mikearcaro/NoHeadResponseMaps; copy archived at https://github.com/elifesciences-publications/NoHeadResponseMaps). The complete dataset that was collected is available upon reasonable request to the corresponding author.

1. 1. Afraz SR
   2. Kiani R
   3. Esteky H

   (2006) Microstimulation of inferotemporal cortex influences face categorization

   Nature 442:692–695.

   https://doi.org/10.1038/nature04982

   • PubMed
   • Google Scholar
2. 1. Afraz A
   2. Boyden ES
   3. DiCarlo JJ

   (2015) Optogenetic and pharmacological suppression of spatial clusters of face neurons reveal their causal role in face gender discrimination

   PNAS 112:6730–6735.

   https://doi.org/10.1073/pnas.1423328112

   • PubMed
   • Google Scholar
3. 1. Aguirre GK
   2. Singh R
   3. D'Esposito M

   (1999) Stimulus inversion and the responses of face and object-sensitive cortical Areas

   NeuroReport 10:189–194.

   https://doi.org/10.1097/00001756-199901180-00036

   • PubMed
   • Google Scholar
4. 1. Arcaro MJ
   2. Schade PF
   3. Vincent JL
   4. Ponce CR
   5. Livingstone MS

   (2017) Seeing faces is necessary for face-domain formation

   Nature Neuroscience 20:1404–1412.

   https://doi.org/10.1038/nn.4635

   • PubMed
   • Google Scholar
5. 1. Arcaro MJ
   2. Schade PF
   3. Livingstone MS

   (2019) Universal mechanisms and the development of the face network: what you see is what you get

   Annual Review of Vision Science 5:341–372.

   https://doi.org/10.1146/annurev-vision-091718-014917

   • PubMed
   • Google Scholar
6. 1. Arcaro MJ
   2. Livingstone MS

   (2017) A hierarchical, retinotopic proto-organization of the primate visual system at birth

   eLife 6:e26196.

   https://doi.org/10.7554/eLife.26196

   • PubMed
   • Google Scholar
7. 1. Baker JF
   2. Petersen SE
   3. Newsome WT
   4. Allman JM

   (1981) Visual response properties of neurons in four extrastriate visual Areas of the owl monkey (Aotus trivirgatus): a quantitative comparison of medial, dorsomedial, dorsolateral, and middle temporal Areas

   Journal of Neurophysiology 45:397–416.

   https://doi.org/10.1152/jn.1981.45.3.397

   • PubMed
   • Google Scholar
8. 1. Bar M

   (2004) Visual objects in context

   Nature Reviews Neuroscience 5:617–629.

   https://doi.org/10.1038/nrn1476

   • PubMed
   • Google Scholar
9. 1. Bar M
   2. Ullman S

   (1996) Spatial context in recognition

   Perception 25:343–352.

   https://doi.org/10.1068/p250343

   • PubMed
   • Google Scholar
10. 1. Bell AH
    2. Hadj-Bouziane F
    3. Frihauf JB
    4. Tootell RB
    5. Ungerleider LG

    (2009) Object representations in the temporal cortex of monkeys and humans as revealed by functional magnetic resonance imaging

    Journal of Neurophysiology 101:688–700.

    https://doi.org/10.1152/jn.90657.2008

    • PubMed
    • Google Scholar
11. 1. Bell AH
    2. Malecek NJ
    3. Morin EL
    4. Hadj-Bouziane F
    5. Tootell RB
    6. Ungerleider LG

    (2011) Relationship between functional magnetic resonance imaging-identified regions and neuronal category selectivity

    Journal of Neuroscience 31:12229–12240.

    https://doi.org/10.1523/JNEUROSCI.5865-10.2011

    • PubMed
    • Google Scholar
12. 1. Bruce C

    (1982) Face recognition by monkeys: absence of an inversion effect

    Neuropsychologia 20:515–521.

    https://doi.org/10.1016/0028-3932(82)90025-2

    • PubMed
    • Google Scholar
13. 1. Chang L
    2. Tsao DY

    (2017) The code for facial identity in the primate brain

    Cell 169:1013–1028.

    https://doi.org/10.1016/j.cell.2017.05.011

    • PubMed
    • Google Scholar
14. 1. Chao LL
    2. Martin A
    3. Lalonde FM
    4. Ungerleider LG
    5. Haxby JV

    (1998) Faces, animals, and animals with obscured faces elicit similar fMRI activation in the ventral object vision pathway

    NeuroImage 7:S350.

    https://doi.org/10.1016/S1053-8119(18)31183-2

    • Google Scholar
15. 1. Cox D
    2. Meyers E
    3. Sinha P

    (2004) Contextually evoked object-specific responses in human visual cortex

    Science 304:115–117.

    https://doi.org/10.1126/science.1093110

    • PubMed
    • Google Scholar
16. 1. Dailey MN
    2. Cottrell GW

    (1999) Organization of face and object recognition in modular neural network models

    Neural Networks 12:1053–1074.

    https://doi.org/10.1016/S0893-6080(99)00050-7

    • PubMed
    • Google Scholar
17. 1. Downing PE
    2. Chan AW
    3. Peelen MV
    4. Dodds CM
    5. Kanwisher N

    (2006) Domain specificity in visual cortex

    Cerebral Cortex 16:1453–1461.

    https://doi.org/10.1093/cercor/bhj086

    • PubMed
    • Google Scholar
18. 1. Fisher C
    2. Freiwald WA

    (2015) Whole-agent selectivity within the macaque face-processing system

    PNAS 112:14717–14722.

    https://doi.org/10.1073/pnas.1512378112

    • Google Scholar
19. 1. Freiwald WA
    2. Tsao DY
    3. Livingstone MS

    (2009) A face feature space in the macaque temporal lobe

    Nature Neuroscience 12:1187–1196.

    https://doi.org/10.1038/nn.2363

    • PubMed
    • Google Scholar
20. 1. Freiwald WA
    2. Tsao DY

    (2010) Functional compartmentalization and viewpoint generalization within the macaque face-processing system

    Science 330:845–851.

    https://doi.org/10.1126/science.1194908

    • PubMed
    • Google Scholar
21. 1. Greene MR

    (2013) Statistics of high-level scene context

    Frontiers in Psychology 4:777.

    https://doi.org/10.3389/fpsyg.2013.00777

    • PubMed
    • Google Scholar
22. 1. Grimaldi P
    2. Saleem KS
    3. Tsao D

    (2016) Anatomical connections of the functionally defined "Face Patches" in the Macaque Monkey

    Neuron 90:1325–1342.

    https://doi.org/10.1016/j.neuron.2016.05.009

    • PubMed
    • Google Scholar
23. 1. Hasson U
    2. Levy I
    3. Behrmann M
    4. Hendler T
    5. Malach R

    (2002) Eccentricity Bias as an organizing principle for human high-order object Areas

    Neuron 34:479–490.

    https://doi.org/10.1016/S0896-6273(02)00662-1

    • PubMed
    • Google Scholar
24. 1. Haxby JV
    2. Ungerleider LG
    3. Clark VP
    4. Schouten JL
    5. Hoffman EA
    6. Martin A

    (1999) The effect of face inversion on activity in human neural systems for face and object perception

    Neuron 22:189–199.

    https://doi.org/10.1016/S0896-6273(00)80690-X

    • PubMed
    • Google Scholar
25. 1. Hong H
    2. Yamins DL
    3. Majaj NJ
    4. DiCarlo JJ

    (2016) Explicit information for category-orthogonal object properties increases along the ventral stream

    Nature Neuroscience 19:613–622.

    https://doi.org/10.1038/nn.4247

    • PubMed
    • Google Scholar
26. 1. Issa EB
    2. DiCarlo JJ

    (2012) Precedence of the eye region in neural processing of faces

    Journal of Neuroscience 32:16666–16682.

    https://doi.org/10.1523/JNEUROSCI.2391-12.2012

    • PubMed
    • Google Scholar
27. 1. Kaiser D
    2. Strnad L
    3. Seidl KN
    4. Kastner S
    5. Peelen MV

    (2014) Whole person-evoked fMRI activity patterns in human fusiform gyrus are accurately modeled by a linear combination of face- and body-evoked activity patterns

    Journal of Neurophysiology 111:82–90.

    https://doi.org/10.1152/jn.00371.2013

    • PubMed
    • Google Scholar
28. 1. Kanwisher N
    2. Tong F
    3. Nakayama K

    (1998) The effect of face inversion on the human fusiform face area

    Cognition 68:B1–B11.

    https://doi.org/10.1016/S0010-0277(98)00035-3

    • PubMed
    • Google Scholar
29. 1. Kanwisher N
    2. Stanley D
    3. Harris A

    (1999) The fusiform face area is selective for faces not animals

    NeuroReport 10:183–187.

    https://doi.org/10.1097/00001756-199901180-00035

    • PubMed
    • Google Scholar
30. 1. Kiani R
    2. Esteky H
    3. Mirpour K
    4. Tanaka K

    (2007) Object category structure in response patterns of neuronal population in monkey inferior temporal cortex

    Journal of Neurophysiology 97:4296–4309.

    https://doi.org/10.1152/jn.00024.2007

    • PubMed
    • Google Scholar
31. Conference

    1. Krizevsky A
    2. Sutskever I
    3. Hinton GE

    (2012)

    ImageNet Classification with Deep Convolutional Neural Networks

    25th International Conference on Neural Information Processing Systems.

    • Google Scholar
32. 1. Leehey S
    2. Carey S
    3. Diamond R
    4. Cahn A

    (1978) Upright and inverted faces: the right hemisphere knows the difference

    Cortex 14:411–419.

    https://doi.org/10.1016/S0010-9452(78)80067-7

    • PubMed
    • Google Scholar
33. 1. Leite FP
    2. Tsao D
    3. Vanduffel W
    4. Fize D
    5. Sasaki Y
    6. Wald LL
    7. Dale AM
    8. Kwong KK
    9. Orban GA
    10. Rosen BR
    11. Tootell RB
    12. Mandeville JB

    (2002) Repeated fMRI using iron oxide contrast agent in awake, behaving macaques at 3 tesla

    NeuroImage 16:283–294.

    https://doi.org/10.1006/nimg.2002.1110

    • PubMed
    • Google Scholar
34. 1. Leopold DA
    2. Bondar IV
    3. Giese MA

    (2006) Norm-based face encoding by single neurons in the monkey inferotemporal cortex

    Nature 442:572–575.

    https://doi.org/10.1038/nature04951

    • PubMed
    • Google Scholar
35. 1. Meeren HK
    2. van Heijnsbergen CC
    3. de Gelder B

    (2005) Rapid perceptual integration of facial expression and emotional body language

    PNAS 102:16518–16523.

    https://doi.org/10.1073/pnas.0507650102

    • PubMed
    • Google Scholar
36. 1. Moeller S
    2. Freiwald WA
    3. Tsao DY

    (2008) Patches with links: a unified system for processing faces in the macaque temporal lobe

    Science 320:1355–1359.

    https://doi.org/10.1126/science.1157436

    • PubMed
    • Google Scholar
37. 1. Moeller S
    2. Crapse T
    3. Chang L
    4. Tsao DY

    (2017) The effect of face patch microstimulation on perception of faces and objects

    Nature Neuroscience 20:743–752.

    https://doi.org/10.1038/nn.4527

    • PubMed
    • Google Scholar
38. 1. Op De Beeck H
    2. Vogels R

    (2000) Spatial sensitivity of macaque inferior temporal neurons

    The Journal of Comparative Neurology 426:505–518.

    https://doi.org/10.1002/1096-9861(20001030)426:4<505::AID-CNE1>3.0.CO;2-M

    • PubMed
    • Google Scholar
39. 1. Peelen MV
    2. Wiggett AJ
    3. Downing PE

    (2006) Patterns of fMRI activity dissociate overlapping functional brain Areas that respond to biological motion

    Neuron 49:815–822.

    https://doi.org/10.1016/j.neuron.2006.02.004

    • PubMed
    • Google Scholar
40. 1. Peelen MV
    2. Downing PE

    (2005) Selectivity for the human body in the fusiform gyrus

    Journal of Neurophysiology 93:603–608.

    https://doi.org/10.1152/jn.00513.2004

    • PubMed
    • Google Scholar
41. 1. Pinsk MA
    2. Arcaro M
    3. Weiner KS
    4. Kalkus JF
    5. Inati SJ
    6. Gross CG
    7. Kastner S

    (2009) Neural representations of faces and body parts in macaque and human cortex: a comparative FMRI study

    Journal of Neurophysiology 101:2581–2600.

    https://doi.org/10.1152/jn.91198.2008

    • PubMed
    • Google Scholar
42. 1. Ponce CR
    2. Xiao W
    3. Schade PF
    4. Hartmann TS
    5. Kreiman G
    6. Livingstone MS

    (2019) Evolving images for visual neurons using a deep generative network reveals coding principles and neuronal preferences

    Cell 177:999–1009.

    https://doi.org/10.1016/j.cell.2019.04.005

    • PubMed
    • Google Scholar
43. 1. Popivanov ID
    2. Jastorff J
    3. Vanduffel W
    4. Vogels R

    (2015) Tolerance of macaque middle STS body patch neurons to shape-preserving stimulus transformations

    Journal of Cognitive Neuroscience 27:1001–1016.

    https://doi.org/10.1162/jocn_a_00762

    • PubMed
    • Google Scholar
44. 1. Rice A
    2. Phillips PJ
    3. Natu V
    4. An X
    5. O'Toole AJ

    (2013) Unaware person recognition from the body when face identification fails

    Psychological Science 24:2235–2243.

    https://doi.org/10.1177/0956797613492986

    • PubMed
    • Google Scholar
45. 1. Rosenfeld SA
    2. Van Hoesen GW

    (1979) Face recognition in the rhesus monkey

    Neuropsychologia 17:503–509.

    https://doi.org/10.1016/0028-3932(79)90057-5

    • PubMed
    • Google Scholar
46. 1. Schwarzlose RF
    2. Baker CI
    3. Kanwisher N

    (2005) Separate face and body selectivity on the fusiform gyrus

    Journal of Neuroscience 25:11055–11059.

    https://doi.org/10.1523/JNEUROSCI.2621-05.2005

    • PubMed
    • Google Scholar
47. 1. Taubert J
    2. Van Belle G
    3. Vanduffel W
    4. Rossion B
    5. Vogels R

    (2015) The effect of face inversion for neurons inside and outside fMRI-defined face-selective cortical regions

    Journal of Neurophysiology 113:1644–1655.

    https://doi.org/10.1152/jn.00700.2014

    • PubMed
    • Google Scholar
48. 1. Tsao DY
    2. Freiwald WA
    3. Tootell RB
    4. Livingstone MS

    (2006) A cortical region consisting entirely of face-selective cells

    Science 311:670–674.

    https://doi.org/10.1126/science.1119983

    • PubMed
    • Google Scholar
49. 1. Vanduffel W
    2. Fize D
    3. Mandeville JB
    4. Nelissen K
    5. Van Hecke P
    6. Rosen BR
    7. Tootell RB
    8. Orban GA

    (2001) Visual motion processing investigated using contrast agent-enhanced fMRI in awake behaving monkeys

    Neuron 32:565–577.

    https://doi.org/10.1016/S0896-6273(01)00502-5

    • PubMed
    • Google Scholar
50. 1. Yin RK

    (1969) Looking at upside-down faces

    Journal of Experimental Psychology 81:141–145.

    https://doi.org/10.1037/h0027474

    • Google Scholar

Author details

1. Michael J Arcaro

   Department of Psychology, University of Pennsylvania, Philadelphia, United States

   Contribution

   Conceptualization, Resources, Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing - original draft, Writing - review and editing

   For correspondence

   marcaro@sas.upenn.edu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-4612-9921

2. Carlos Ponce

   Department of Neuroscience, Washington University in St. Louis, St. Louis, United States

   Contribution

   Conceptualization, Software, Visualization, Methodology

   Competing interests

   No competing interests declared

3. Margaret Livingstone

   Department of Neurobiology, Harvard Medical School, Boston, United States

   Contribution

   Conceptualization, Resources, Data curation, Formal analysis, Supervision, Funding acquisition, Investigation, Visualization, Writing - original draft, Writing - review and editing

   Competing interests

   No competing interests declared

• 3,548

  views

• 452

  downloads

• 18

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.